Medical image processing device and medical observation system

ABSTRACT

A medical image processing device includes: an acquisition unit configured to acquire an image signal obtained by capturing a subject image; and an image processor configured to set, in the image signal, a first area for displaying an image, and a second area having a smaller average luminance than the first area, set the second area on a screen according to a depth of field at a time of capturing the subject image.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority from Japanese Application No.2020-035368, filed on Mar. 2, 2020, the contents of which areincorporated by reference herein in its entirety.

BACKGROUND

The present disclosure relates to a medical image processing device anda medical observation system.

Medical observation systems are known that capture enlarged images ofminute parts and display the captured images on a monitor when surgeryis being performed on the minute parts of the brain, heart, or the like,of a patient who is an object under observation (see Japanese Laid-openPatent Publication No. 2016-42982, for example). In this medicalobservation system, an imaging unit has a zoom function and a focusfunction.

SUMMARY

In the foregoing medical observation systems, when imaging is performedin a state where the zoom factor is raised and the depth of field isthen shallow, for example, the whole of the screen is not in focus, anda portion of the screen readily enters a blurred state. When a user suchas a physician observes this kind of captured image via a monitor, ablurred area is also visible. Under these circumstances, a techniquethat enables the generation of a higher visibility image has been indemand.

There is a need for a medical image processing device and a medicalobservation system that enable high-visibility images to be generated.

According to one aspect of the present disclosure, there is provided amedical image processing device including: an acquisition unitconfigured to acquire an image signal obtained by capturing a subjectimage; and an image processor configured to set, in the image signal, afirst area for displaying an image, and a second area having a smalleraverage luminance than the first area, set the second area on a screenaccording to a depth of field at a time of capturing the subject image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram schematically illustrating a medical observationsystem according to an embodiment;

FIG. 2 is a block diagram illustrating a function configuration of amedical observation device;

FIG. 3 is a diagram illustrating a (first) display example of an imagegenerated by an image processor and displayed on a display device;

FIG. 4 is a diagram illustrating a (second) display example of an imagegenerated by the image processor and displayed on the display device;and

FIG. 5 is a diagram illustrating a (third) display example of an imagegenerated by the image processor and displayed on the display device.

DETAILED DESCRIPTION

A mode for carrying out the present disclosure (hereinafter called “theembodiment”) will be described hereinbelow with reference to theattached drawings.

FIG. 1 is a diagram schematically illustrating a medical observationsystem according to an embodiment. A medical observation system 1illustrated in FIG. 1 includes a medical observation device 2 and adisplay device 3.

The medical observation device 2 includes a microscope device 4 and acontrol device 5. The microscope device 4 has an imaging device functionfor imaging objects under observation and acquiring image signals. Thecontrol device 5 has a medical image processing device function forperforming image processing on the image signals captured by themicroscope device 4. The medical observation device 2 according to thepresent embodiment is a surgical microscope.

The display device 3 receives, from the control device 5, a displayimage signal generated by the control device 5 and displays an imagecorresponding to the image signal. The display device 3 has a lightemission amount control unit 31 for controlling the amount of lightemitted for each area of a video displayed. The display device 3 isconfigured using a display panel configured from liquid crystals ororganic electroluminescent (EL) diodes. When the display panel is aliquid-crystal display panel, the light emission amount control unit 31controls the amount of emission light of a backlight configured from aplurality of light-emitting diodes (LEDs) arranged on the back side ofthe display panel. Furthermore, when the display panel is configuredfrom organic EL diodes, the light emission amount control unit 31controls the light emission amounts of self-light emitting elements.

The outer appearance of the microscope device 4 will now be described.The microscope device 4 has a microscope part 6 that captures enlargedimages of the microstructure of an object under observation; a supportpart 7 that supports the microscope part 6; and a base part 8 that holdsthe proximal end of the support part 7 and that incorporates the controldevice 5.

The microscope part 6 has a tubular section with a cylindrical shape. Acover glass is provided to the aperture side at the lower end section ofthe body section (not illustrated). The tubular section may be graspedby the user and is of a size enabling the user to move the tubularsection while grasping same when changing the imaging field of themicroscope part 6. Note that the shape of the tubular section is notlimited to being cylindrical and may instead have a polygonal tubularshape.

An arm section of the support part 7 has a plurality of links, andadjacent links are turnably coupled to each other via a joint section. Atransmission cable for transmitting various signals between themicroscope part 6 and the control device 5, and a light guide fortransmitting illumination light generated by the control device 5 to themicroscope part 6 pass through a hollow section formed inside thesupport part 7.

FIG. 2 is a block diagram illustrating a function configuration of themedical observation device 2. First, the function configuration of themicroscope device 4 will be described. The microscope device 4 includesa lens unit 41, a lens drive unit 42, a diaphragm 43, a diaphragm driveunit 44, a detection unit 45, an imaging unit 46, an arm drive unit 47,an input unit 48, a communications unit 49, and a control unit 4 a.

The lens unit 41 is configured using a plurality of lenses capable ofmoving along an optical axis and forms a condensed subject image on theimaging surface of the imaging element of the imaging unit 46. The lensunit 41 has a focusing lens 411 for adjusting the focal point and a zoomlens 412 for changing the angle of view. The focusing lens 411 and zoomlens 412 are each configured using one or a plurality of lenses.

The lens drive unit 42 has an actuator that operates the zoom lens and adriver that drives the actuator, under the control of the control unit 4a.

The diaphragm 43 is provided between the lens unit 41 and the imagingunit 46 and adjusts the amount of light of the subject image from thelens unit 41 toward the imaging unit 46 under the control of the controlunit 4 a.

The diaphragm drive unit 44 adjusts the aperture value (also called theF-number) by operating the diaphragm 43, under the control of thecontrol unit 4 a.

The detection unit 45 has two position sensors that detect therespective positions of the focusing lens 411 and the zoom lens 412, andan encoder that detects the aperture value of the diaphragm 43, or thelike. The detection unit 45 outputs, to the control unit 4 a, theposition of the zoom lens 412 and the aperture value of the diaphragm 43that are detected.

The imaging unit 46 has an imaging element that generates a capturedimage (an analog signal) by forming the subject image that has beencondensed by the lens unit 41, and a signal processor that performssignal processing such as noise removal and A/D conversion, and thelike, on the image signal (analog signal) from the imaging element. Theimaging element is configured using an image sensor such as acharge-coupled device (CCD) or a complementary metal-oxide semiconductor(CMOS). Note that the imaging unit 46 may have two imaging elements. Inthis case, the imaging unit 46 is capable of generating athree-dimensional image (3D image).

The arm drive unit 47 operates each of a plurality of arms of thesupport part 7, under the control of the control unit 4 a. Morespecifically, the arm drive unit 47 has an actuator provided at thejoint sections between the arms, and a driver that drives the actuator.

The input unit 48 accepts inputs such as the operation signals of thelens unit 41 and the operation signals of the arm of the support part 7.The input unit 48 has a plurality of switches and buttons, and the like,provided in positions, on the lateral surface of the tubular section ofthe microscope part 6, which enable operation in a state where the useris grasping the microscope part 6.

The communications unit 49 is an interface that communicates with thecontrol device 5. The communications unit 49 transmits image signals(digital signals) generated by the imaging unit 46 to the control device5 and receives control signals from the control device 5.

The control unit 4 a controls the operation of the microscope device 4in cooperation with a control unit 55 of the control device 5. Thecontrol unit 4 a causes the microscope device 4 to operate on the basisof operation instruction signals received through inputs by the inputunit 48 and operation instruction signals transmitted from the controlunit 55 of the control device 5. In the present embodiment, a signal tooperate the arm in order to move the imaging field of the microscopedevice 4 is received from the control unit 55 of the control device 5.

The control unit 4 a is configured by using at least any one processoramong a central processing unit (CPU), a field programmable gate array(FPGA), and an application specific integrated circuit (ASIC), or thelike.

Next, the function configuration of the control device will bedescribed. The control device 5 includes a communications unit 51, animage processor 52, an input unit 53, a light source unit 54, thecontrol unit 55, and a storage unit 56. The control device 5 is capableof setting a first mode in which a normal video signal is generated andoutputted, and a second mode in which a video signal, which has a firstarea for displaying video and a second area having a smaller averageluminance than the first area, is generated and outputted.

The communications unit 51 acquires image signals that have beencaptured by the microscope device 4 and transmitted via a transmissioncable. The image signals contain image-related information such as again adjustment value, the focusing lens position, the zoom lensposition, the shutter speed, and the aperture value, during imaging.

The image processor 52 has an area setting unit 521 and a maskgeneration unit 522.

When the control device 5 has set the second mode, the area setting unit521 sets, for the image signal acquired by the communications unit 51and according to the depth of field, a first area for displaying animage and a second area having a smaller average luminance than thefirst area. The area setting unit 521 refers to the storage unit 56 andsets a second area which is sized according to the zoom factor and/oraperture value corresponding to the detection results by the detectionunit 45. More specifically, the area setting unit 521 sets the surfacearea of the second area to be larger as the depth of field becomesshallower. The depth of field becomes shallower if the zoom factor isincreased and shallower if the aperture value is reduced. The zoomfactor is decided on the basis of the positions of the zoom lens 412 andfocusing lens 411, which are detected by the detection unit 45.

When the control device 5 has set the second mode, the mask generationunit 522 generates a mask that corresponds to the second area set by thearea setting unit 521 and overlays the generated mask on the imagesignal.

The image processor 52 also generates a display image signal byperforming various signal processing on the image signal acquired by thecommunications unit 51 and outputs the generated image signal to thedisplay device 3. Examples of specific image processing may includewell-known image processing such as processing to detect the brightnesslevel of the image signal, gain adjustment, interpolation processing,color correction processing, color enhancement processing, and contourenhancement processing. The image processor 52 is configured using atleast any one processor among a CPU, an FPGA, and an ASIC, or the like.

FIGS. 3 and 4 are diagrams illustrating an example of an image generatedby the image processor 52 and displayed on the display device 3. FIGS. 3and 4 schematically illustrate a situation where a body part of apatient which is being operated on is viewed by changing the zoom factorand the field of view. An image 101 illustrated in FIG. 3 has a firstarea 111 disposed in the center of the screen and a second area 112disposed on both the left and right sides of the first area 111.Furthermore, an image 102 illustrated in FIG. 4 has a first area 121disposed in the center of the screen and a second area 122 disposed onboth the left and right sides of the first area 121. In the image 101illustrated in FIG. 3 , whereas a part from which a portion of tissue isremoved using a surgical instrument is visible in the upper right-handsection of the first area 111, in the image 102 illustrated in FIG. 4 ,the same part is visible in the center of the first area 121 where thesame part is enlarged. Obviously, from comparing the same part, theimage 102 is an image with a large zoom factor and has a shallow depthof field. When the second area 112 illustrated in FIG. 3 is comparedwith the second area 122 illustrated in FIG. 4 , the surface area of thesecond area 122 of the image 102, which has a relatively shallow depthof field, is smaller than the surface area of the second area 112 of theimage 101. Thus, by increasing the surface area of the second area asthe depth of field becomes shallower, a mask may be overlaid on parts inthe vicinity of which there is likely to be blurring as the depth offield becomes shallower, thereby enabling an affected area, or the like,which the user would like to view to be displayed more clearly.

Note that the shape of the boundary between the first and second areasmay also be a noncircular shape. The color of the second area may alsobe a color other than black and may not be monochrome.

The input unit 53 receives inputs of various information. The input unit53 is configured using a user interface such as a keyboard, a mouse, atouch panel, or a foot switch. Note that the input unit 53 may alsofulfill at least some of the functions of the input unit 48 of themicroscope device 4.

The light source unit 54 generates illumination light that is suppliedto the microscope device 4 via a light guide. The light source unit 54is configured using, for example, a solid state light-emitting elementsuch as a light-emitting diode (LED) or a laser diode (LD), or a laserlight source, a xenon lamp, or a halogen lamp, or the like.

The control unit 55 has a brightness control unit 551 and a displaycontrol unit 552.

In order to set the brightness level of the image signal captured by themicroscope device 4 at a predetermined brightness level, the brightnesscontrol unit 551 controls the gain adjustment of the imaging unit 46,the shutter speed, the gain adjustment performed by the image processor52, and the amount of illumination light generated by the light sourceunit 54, and the like.

The display control unit 552 controls the display of the display device3. In addition to normal display control, the display control unit 552outputs, to the display device 3, a signal for controlling the lightemission amount of the backlight of the display device 3. Note that,when the display panel of the display device 3 is configured fromorganic EL diodes, the display control unit 552 outputs, to the displaydevice 3, a signal for controlling the light emission amount of theself-light emitting elements.

The control unit 55 controls the operation of the control device 5 andperforms centralized control of the operation of the medical observationdevice 2 in cooperation with the control unit 4 a of the microscopedevice 4. When the control device 5 is set to the second mode, thecontrol unit 55 performs autofocus (AF) and auto-exposure (AE) byexcluding the second area from the target of the AF and AE. Thus, it ispossible to prevent an erroneous AF or AE operation due to same beingaffected by the screen edge sections. The control unit 55 is configuredusing at least any one processor among a CPU, an FPGA, and an ASIC, orthe like. Note that the image processor 52 and the control unit 55 maybe configured using a common processor.

The storage unit 56 stores zoom factors and aperture values, and thesurface area of the second area overlaid on the image displayed on thedisplay device 3, in association with each other. The relationshipbetween the zoom factor, the aperture value, and the surface area of thesecond area is such that the surface area of the second area becomeslarger as the depth of field becomes shallower, as described withreference to FIGS. 3 and 4 . The depth of field becomes shallower as thezoom factor increases and shallower as the aperture value decreases.Hence, the storage unit 56 stores a relationship where the surface areaof the second area becomes larger as the zoom factor becomes larger, andwhere the surface area of the second area becomes larger as the aperturevalue becomes smaller. Note that the relationship stored by the storageunit 56 may also be a relationship where the size of the mask variesincrementally according to the zoom factor and the aperture value.

The storage unit 56 stores various programs enabling the control device5 to operate in the position of the zoom lens 412 and temporarily storesdata during the arithmetic processing by the control device 5. Thestorage unit 56 is configured using a read-only memory (ROM) or arandom-access memory (RAM), or the like.

When a user such as a physician uses the medical observation system 1with the foregoing configuration to perform surgery on the head, or thelike, of a patient, the user performs the surgery while viewing imagesdisplayed by the display device 3. When the display device 3 displays 3Dimages, the user views the display device 3 by wearing 3D glasses.

According to one embodiment described hereinabove, when a first area fordisplaying images, and a second area having a smaller average luminancethan the first area are set in an image signal which is obtained byimaging a minute part of an object under observation, because thesurface area of the second area on the screen is set to be larger as thedepth of field when capturing the image becomes shallower, on-screencontrast is obtained, and a high-visibility image may be generated.

Furthermore, according to the present embodiment, because the surfacearea of the second area is made larger as the zoom factor increases andas the aperture value decreases, it is possible to reduce screenblurring irrespective of the zoom factor or aperture value, enabling theuser to gaze at the area which is in focus.

Further, according to the present embodiment, by providing a mask area,it is possible to provide images that cause minimal discomfort to userswho are accustomed to a surgical microscope of a certain optical system.

Although a mode for carrying out the present disclosure has beendescribed hereinabove, the present disclosure should not be limited toor by the foregoing one embodiment. For example, the light emissionamount control unit 31 of the display device 3 may perform control toreduce the light emission amount of the second area and increase thelight emission amount of the area where the image captured by themicroscope device 4 is visible. Thus, visibility may be improved.

Furthermore, the display control unit 552 may transmit a control signalto the light emission amount control unit 31 of the display device 3 toreduce the light emission amount of the area corresponding to the secondarea and increase the light emission amount of the area where the imagecaptured by the microscope device 4 is visible. FIG. is a diagramschematically illustrating a display example of the display device 3 inthis case. A second area 132 of an image 103 illustrated in FIG. 5 isnot masked, and the average light emission amount of the second area 132is smaller than the average light emission amount of a first area 131.In this case, by only implementing control of the light emission amountand without overlaying a mask, it is possible to obtain the same effectas when a mask is overlaid.

In addition, the image processor 52 may generate an image signal so thatthe average light emission amount of the second area is smaller than theaverage light emission amount of the first area and transmit this imagesignal to the display device 3.

Furthermore, an on-screen display (OSD) function for displaying variousinformation relating to the medical observation system 1 on the displaydevice 3 may be provided, and such information may be displayed in thesecond area. Thus, the various information does not need to be displayedoverlaid on video in the first area. In addition, as long as the variousinformation is displayed darker in the second area than in the firstarea, the visibility of the first area is not hindered.

Furthermore, the manner in which images are displayed by the displaydevice 3 may be a display mode in which the center of the first area iseccentric from the center of the screen and the second area hasleft-right asymmetry.

The medical observation device according to the present disclosure mayalso be an endoscope or an external scope.

Note that this technology may also adopt the following configurations.

(1) A medical image processing device including:

-   -   an acquisition unit configured to acquire an image signal        obtained by capturing a subject image; and    -   an image processor configured to        -   set, in the image signal, a first area for displaying an            image, and a second area having a smaller average luminance            than the first area,        -   set the second area on a screen according to a depth of            field at a time of capturing the subject image.            (2) The medical image processing device according to (1),            wherein the image processor is configured to set a surface            area of the second area on the screen to be larger as the            depth of field at the time of capturing the subject image            becomes shallower.            (3) The medical image processing device according to (1) or            (2), wherein the image processor is configured to set a            surface area of the second area to be larger as a zoom            factor at a time of capturing the subject image becomes            larger.            (4) The medical image processing device according to any one            of (1) to (3), wherein the image processor is configured to            set a surface area of the second area to be larger as an            aperture value at a time of capturing the subject image            becomes smaller.            (5) The medical image processing device according to any one            of (1) to (4), wherein the image processor is configured to            generate a mask corresponding to the second area, and            overlay the mask on the image signal.            (6) The medical image processing device according to any one            of (1) to (5), further including a controller configured to            output, to a display, a control signal that renders an            average light emission amount of the second area smaller            than an average light emission amount of the first area.            (7) The medical image processing device according to (1),            wherein the image processor is configured to generate the            image signal to cause the display configured to control an            light emission amount in each area according to the image            signal to display such that an average light emission amount            of the second area is smaller than an average light emission            amount of the first area.            (8) The medical image processing device according to (5) or            (6), wherein the image processor is configured to set, in            the second area, a display area for displaying information            relating to an imaging device configured to generate the            image signal and to the medical image processing device.            (9) The medical image processing device according to any one            of (1) to (8), further including a controller configured to            perform autofocus and auto-exposure by excluding the second            area.            (10) A medical observation system, including:    -   an imaging device configured to capture a subject image;    -   circuitry configured to        -   acquire an image signal from the imaging device,        -   set, in the image signal, a first area for displaying an            image, and a second area having a smaller average luminance            than the first area; and        -   set the second area on a screen according to a depth of            field at a time of capturing the subject image; and    -   a display configured to display an image based on a display        image signal generated by the circuitry.

According to the present disclosure, high-visibility images may begenerated.

Although the disclosure has been described with respect to specificembodiments for a complete and clear disclosure, the appended claims arenot to be thus limited but are to be construed as embodying allmodifications and alternative constructions that may occur to oneskilled in the art that fairly fall within the basic teaching herein setforth.

What is claimed is:
 1. A medical image processing device comprising: anacquisition circuit configured to acquire an image signal obtained bycapturing a subject image; and an image processor configured to set, inthe image signal, a first area for displaying an image, and a secondarea having a smaller average luminance than the first area according toa depth of field at a time of capturing the subject image, wherein anaverage light emission amount of the second area is less than an averagelight emission amount of the first area but greater than zero, and oncondition that the depth of field changes, increase the second area on ascreen as the depth of field becomes shallower.
 2. The medical imageprocessing device according to claim 1, wherein the image processor isconfigured to generate a mask corresponding to the second area, andoverlay the mask on the image signal.
 3. The medical image processingdevice according to claim 2, wherein the image processor is configuredto set, in the second area, a display area for displaying informationrelating to an imaging device, configured to generate the image signal,and to the medical image processing device.
 4. The medical imageprocessing device according to claim 1, further comprising a controlcircuit configured to output, to a display, a control signal thatrenders the average light emission amount of the second area smallerthan the average light emission amount of the first area but greaterthan zero.
 5. The medical image processing device according to claim 4,wherein the average light emission amount of the second area is greaterthan zero.
 6. The medical image processing device according to claim 1,wherein the image processor is configured to generate the image signalto cause the screen, configured to control a light emission amount ineach area according to the image signal, to display the image such thatthe average light emission amount of the second area is smaller than theaverage light emission amount of the first area but greater than zero.7. The medical image processing device according to claim 6, wherein theaverage light emission amount of the second area is greater than zero.8. The medical image processing device according to claim 1, furthercomprising a control circuit configured to perform autofocus andauto-exposure by excluding the second area.
 9. A medical observationsystem, comprising: an imaging device configured to capture a subjectimage; circuitry configured to acquire an image signal from the imagingdevice, set, in the image signal, a first area for displaying an image,and a second area having a smaller average luminance than the first areaaccording to a depth of field at a time of capturing the subject image,wherein an average light emission amount of the second area is less thanan average light emission amount of the first area but greater thanzero; and on condition that the depth of field changes, increase thesecond area on a screen as the depth of field becomes shallower; and adisplay configured to display an image based on a display image signalgenerated by the circuitry.
 10. The medical observation system accordingto claim 9, wherein the circuitry is further configured to performautofocus and auto-exposure by excluding the second area.
 11. Themedical observation system according to claim 9, wherein the circuitryis further configured to output, to the display, a control signal thatrenders the average light emission amount of the second area smallerthan the average light emission amount of the first area but greaterthan zero.
 12. The medical observation system according to claim 9,wherein the circuitry is further configured to generate the image signalto cause the screen, configured to control a light emission amount ineach area according to the image signal, to display the image such thatthe average light emission amount of the second area is smaller than theaverage light emission amount of the first area but greater than zero.13. A medical image adjusting method, comprising: setting, in an imagesignal received from an imaging device capturing a subject image, afirst area for displaying an image, and a second area having a smalleraverage luminance than the first area according to a depth of field at atime of capturing the subject image, wherein an average light emissionamount of the second area is less than an average light emission amountof the first area but greater than zero; and on condition that the depthof field changes, increasing the second area on a screen as the depth offield becomes shallower.
 14. The medical image adjusting methodaccording to claim 13, further comprising performing autofocus andauto-exposure by excluding the second area.
 15. The medical imageadjusting method according to claim 13, further comprising: generating amask corresponding to the second area, and overlaying the mask on theimage signal.
 16. The medical image adjusting method according to claim13, further comprising outputting, to a display, a control signal thatrenders the average light emission amount of the second area smallerthan the average light emission amount of the first area but greaterthan zero.
 17. The medical image adjusting method according to claim 13,further comprising generating the image signal to cause the screen,configured to control a light emission amount in each area according tothe image signal, to display the image such that the average lightemission amount of the second area is smaller than the average lightemission amount of the first area but greater than zero.